Skip to content

docs(examples): add nemoguards cache configuration example #1459

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 2 commits into from
Oct 29, 2025
Merged

docs(examples): add nemoguards cache configuration example #1459

merged 2 commits into from
Oct 29, 2025

Conversation

@Pouyanpi
Copy link
Collaborator

@Pouyanpi Pouyanpi commented Oct 17, 2025 •
edited
Loading

Description

This is PR 5/5 (final) in the NeMoGuards caching feature stack.

Test Plan

results:

======================================================================
MODEL CACHE CONFIGURATION
======================================================================
✅ content_safety: maxsize=10000, track_stats=True
✅ topic_control: maxsize=10000, track_stats=True
✅ jailbreak_detection: maxsize=10000, track_stats=True

======================================================================
GENERATING 1000 TEST PROMPTS
======================================================================
Generated 1000 prompts (100 unique × 10 repetitions)

======================================================================
PROCESSING PROMPTS
======================================================================
Processed 100/1000 prompts - Hits: 270, Misses: 30, Errors: 0
Processed 200/1000 prompts - Hits: 540, Misses: 60, Errors: 0
Processed 300/1000 prompts - Hits: 810, Misses: 90, Errors: 0
Processed 400/1000 prompts - Hits: 1080, Misses: 120, Errors: 0
Processed 500/1000 prompts - Hits: 1350, Misses: 150, Errors: 0
Processed 600/1000 prompts - Hits: 1620, Misses: 180, Errors: 0
Processed 700/1000 prompts - Hits: 1890, Misses: 210, Errors: 0
Processed 800/1000 prompts - Hits: 2160, Misses: 240, Errors: 0
Processed 900/1000 prompts - Hits: 2430, Misses: 270, Errors: 0
Processed 1000/1000 prompts - Hits: 2700, Misses: 300, Errors: 0

Completed processing 1000 prompts

======================================================================
CACHE STATISTICS FROM LOGS
======================================================================
content_safety       - Hits:  900, Misses:  100, Hit Rate:  90.0%
topic_safety         - Hits:  900, Misses:  100, Hit Rate:  90.0%
jailbreak            - Hits:  900, Misses:  100, Hit Rate:  90.0%

======================================================================
CACHE STATISTICS FROM CACHE OBJECTS
======================================================================
content_safety       - Size: 100/10000, Hits: 900, Misses: 100, Hit Rate: 90.00%
topic_control        - Size: 100/10000, Hits: 900, Misses: 100, Hit Rate: 90.00%
jailbreak_detection  - Size: 100/10000, Hits: 900, Misses: 100, Hit Rate: 90.00%

======================================================================
VALIDATING traces.jsonl
======================================================================
Total LLM spans in traces: 3000
Cached LLM spans (llm.cache.hit=true): 2700

content_safety       - Total: 1000, Cached:  900, Hit Rate:  90.0%
topic_safety         - Total: 1000, Cached:  900, Hit Rate:  90.0%
jailbreak            - Total: 1000, Cached:  900, Hit Rate:  90.0%

Traces file: /Users/prezakhani/git/nemoguardrails-project/nemo-guardrails-review/.traces/trace.jsonl

======================================================================
SUMMARY
======================================================================
✅ Processed 1000 prompts with input rails (content safety, topic safety, jailbreak)
✅ Total cache hits: 2700
✅ Total cache misses: 300
✅ Overall cache hit rate: 90.0%
✅ Traces validated for llm.cache.hit attribute across all safety checks

This was referenced Oct 17, 2025
Merged
Merged
@Pouyanpi Pouyanpi force-pushed the feat/cache-jailbreak-detection branch from 6a7474e to 630eafa Compare October 17, 2025 10:35
@codecov-commenter
Copy link

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@Pouyanpi Pouyanpi force-pushed the feat/cache-jailbreak-detection branch from 630eafa to 9dbb323 Compare October 17, 2025 10:40
@Pouyanpi Pouyanpi force-pushed the docs/cache-nemoguards-example branch from 9495808 to 27685a2 Compare October 17, 2025 10:41
This was referenced Oct 17, 2025
Merged
Merged
@Pouyanpi Pouyanpi added this to the v0.18.0 milestone Oct 17, 2025
@Pouyanpi Pouyanpi self-assigned this Oct 17, 2025
tgasser-nv
tgasser-nv reviewed Oct 17, 2025
View reviewed changes
Copy link
Collaborator

@tgasser-nv tgasser-nv left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need a description of how this config is different to the regular hero-workflow one (i.e. caching) and how they're configured. I can update the PR as you have so many others on your plate

@Pouyanpi Pouyanpi force-pushed the feat/cache-jailbreak-detection branch from 9dbb323 to d38eb0b Compare October 19, 2025 10:26
Base automatically changed from feat/cache-jailbreak-detection to develop October 19, 2025 10:38
@Pouyanpi Pouyanpi force-pushed the docs/cache-nemoguards-example branch from 9649c8a to 9cbb265 Compare October 20, 2025 10:54
@Pouyanpi Pouyanpi force-pushed the docs/cache-nemoguards-example branch from 9cbb265 to ebb5f09 Compare October 20, 2025 10:55
@github-actions
Copy link
Contributor

Documentation preview

https://nvidia-nemo.github.io/Guardrails/review/pr-1459

@Pouyanpi
Copy link
Collaborator Author

@greptileai

@Pouyanpi Pouyanpi requested a review from tgasser-nv October 21, 2025 13:10
greptile-apps[bot]
greptile-apps bot reviewed Oct 21, 2025
View reviewed changes
Copy link

@greptile-apps greptile-apps bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

3 files reviewed, no comments

Edit Code Review Agent Settings | Greptile

tgasser-nv
tgasser-nv approved these changes Oct 29, 2025
View reviewed changes
Copy link
Collaborator

@tgasser-nv tgasser-nv left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM thanks for the updates

@Pouyanpi Pouyanpi merged commit 28826ec into develop Oct 29, 2025
18 checks passed
@Pouyanpi Pouyanpi deleted the docs/cache-nemoguards-example branch October 29, 2025 16:28
@tgasser-nv tgasser-nv mentioned this pull request Oct 29, 2025
Open
4 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@tgasser-nv tgasser-nv tgasser-nv approved these changes

+1 more reviewer

@greptile-apps greptile-apps[bot] greptile-apps[bot] left review comments

Reviewers whose approvals may not affect merge requirements

Assignees

@Pouyanpi Pouyanpi

Labels

None yet

Projects

None yet

Milestone

v0.18.0

Development

Successfully merging this pull request may close these issues.

4 participants

@Pouyanpi @codecov-commenter @tgasser-nv